I get that Element needs to focus on one use case in order to pay the bills, but there's a lot of room for applications that have barely been explored - calendars, contacts, games, social media, heck - you could build a federated gitlab-like collaboration site with truly federated data. You get multi-device sync, e2e encryption, and identity management out of the box.

Thanks for putting this together, I hope there will be a gold rush as people realize how much low-hanging fruit there is!

I'm interested in this just to keep my devices sync'd up. I know Samsung/Apple/MS etc all have syncing services, but I really really want to get all my data out of their ecosystems.

Just a distributed calendar app would be pretty amazing. No more iCal subscription/export/whatevers, 'just' federate the calendar between devices, and arbitrarily share it with anyone else.

I have no clue where to start with building an app on these abstractions. Does anyone have any pointers? Documentation to read? High level overviews? Etc.?

Matrix-CRDT posted here should abstract most of the nitty gritty stuff, but there are no shortcuts to designing with CRDTs so familiarizing yourself with those is important.

Other than that, just start playing and asking questions! The #matrix:matrix.org room is full of friendly and helpful people and I'm sure you'd be able to get answers.

[0]https://arxiv.org/pdf/2011.06488.pdf

My biased suggestion would be to see how far you get with storing the data to be synced in SyncedStore, and connecting Matrix-CRDT and y-indexeddb for local storage.

Like Matrix, JMAP is commonly misunderstood, with people thinking it’s about email (it’s replacing IMAP, which is Internet Mail Access Protocol, right?), but it’s actually fundamentally an object synchronisation protocol, with the core in RFC 8620 being domain-neutral, and the email model completely separate in RFC 8621. (IMAP has also shifted a bit over time from being about mail with proper synchronisation being an unsound and secondary concern, to nailing down sound object synchronisation in extensions. But JMAP does it better.)

Matrix is about decentralisation while JMAP is client-server, but you can probably reuse much or all of the underlying data models, just as JMAP for Calendars and JMAP for Contacts are establishing JMAP-independent data models models first.

100% agree! Web3 is really only tackling monetary instruments, which is interesting but very narrow. The majority of apps people need don't require global consensus, and it's too slow and expensive anyway. Yet, there's so much interesting stuff that can be decentralized/federated.

> Matrix takes care of a lot of stuff: Authentication, E2EE, federation, hosting, etc. - so I can focus on the client.

This is great to hear! What else besides CRDTs do you think is needed in order for matrix to be a platform for general purpose apps like this?

- What happens with E2EE when there are key exchange problems? (i.e. a server goes down and the account is moved)

- What happens when someone redacts a message? I assume you'd want to disable redactions for these rooms or you'll end up with broken update trees!

- How do you communicate clearly that any data you add is in the event chain forever? In Word and GDocs you can remove old revisions of a document, but I don't think you can in this system? That could be a feature, of course!

- What's the performance impact of such a system on a server editing reasonably large documents? If twenty people each edit ten to twenty documents you end up with quite a large set of state, especially over time!

- If you'd enable E2EE for the Matrix client that you pass to Matrix-CRDT, key management is covered in the same way that Matrix does it normally. If key sharing with a particular user is broken, then you won't see updates from that user anymore and v.v. Basically, the transport between you and that user is broken. As CRDTs such as Yjs are designed specifically to work Local-first, at the core there is no assumption that all clients should always be connected to each other. Once the clients are able to communicate again, potential conflicts would be "resolved" according to the CRDT design.

- Redacting: yes, basically you need to trust clients not to fiddle with messages (I think this is fair as you trust them to work with you on the same data already)

- UX / communication: good question! Technically would be possible to purge old (deleted data), but I think this is still something we need to explore together while we start to see more mature software built on these technologies.

- Kevin already answered your performance questions. Matrix-CRDT makes an additional optimization so that not the entire history of the Matrix room needs to be retrieved (see "Snapshots" in the readme)

Additionally, exposing the CRDT stream would allow for reactive index building for searches on the data (ie. timely dataflow operations to aggregate an index).

Relm [2] even models a 3d world using Yjs. I don't necessarily recommend doing this as 3d applications usually produce a lot more actions per minute than text applications. This required some workarounds and deep knowledge of how Yjs' optimizations work. But it's definitely possible.

[1]: https://blog.kevinjahns.de/are-crdts-suitable-for-shared-edi... [2]: https://www.relm.us/

I have said it before and I will say it again, I think this year is going to be the year of Yjs. The ecosystem around it is brilliant and I believe it will become the leading CRDT framework (it probably already is).

Happy to see you using TipTap too for the demo, a brilliant rich text editor framework (with support for Vue.js, React and Svelte) built on top ProseMirror with first class Yjs support.

https://tiptap.dev

The TipTap guys (Philipp and Hans) along with Kevin Jahns of yJS have stated the y-collective inorder to centralise funding and organising of the Yjs ecosystem:

https://opencollective.com/y-collective

Personally I'm excited about the us of Yjs and CRDTs outside of just rich text editing, I think there is great potential to build a distributed offline enabled database targeting mobile and PWAs with it. Something like PouchDB but with automatic conflict resolution.

Finally there is a Rust port in development to improve the (already very good) performance and make it cross platform with other languages.

https://github.com/y-crdt/y-crdt

BTW, we are already tinkering on some interesting stuff with the Rust port, too. :-)

Take a look at https://concordant.io

Some nice things:

- Anyone can host their own Matrix homeserver, which can then sync its data within a federation, allowing users pretty fine-grained control over what data they share and with whom.

- Add end-to-end encryption, and it becomes even better: data stored somewhere in the cloud (i.e. the user doesn't have to manage a server), but the server knows almost nothing about what you've stored aside from some metadata.

I have a prototype running that uses Matrix-CRDT and Matrix E2EE, and it worked great. It's still a bit of a hassle to set up though (mainly, configuring the matrix sdk correctly), I hope to make that easier later this year.

Instead, I want to encourage you to build an API that others can use to efficiently store shared data. Feel free to ping me if you need input.

- Kevin (Author of Yjs)

Can you expand on what you mean by this?

> an API that others can use to efficiently store shared data

What would you expect this API look like in a bit more detail? Would it be able to abstract any of the underlying CRDT logic? Would it just be a raw stream of authenticated messages with partial ordering? Something in-between?

The matrix-crdt works really well. To reduce overloading the Matrix server with many small messages (each single keystroke produces an update message), it stores merged updates in the DAG after a short debounce. The optional WebRTC extension allows you to distribute messages immediately "of the chain", so you don't notice the debounce.

After a time the message-log gets pretty huge. So in matrix-crdt, a random client will eventually store a "snapshots" of the current state in the DAG and removes old entries. This way, new clients don't need to download the huge message-log.

It would be nice if there was a possibility to create a server-component that does the merging.

(Btw, all credit to the above approach goes to Yousef)

Now, there might be a better solution to store CRDT data in the Matrix DAG - the developers probably know best and might be able to expose some hidden API that would make everything even more efficient.

I'm just asking that instead of creating yet another CRDT and integrating it into Matrix, open up this space, provide better APIs, and let others integrate their CRDTs.

> Would it be able to abstract any of the underlying CRDT logic?

Modern shared-editing frameworks don't require you to think about internal logic. They just set some requirements on the ordering of update messages. CRDTs in particular don't care in which order you transmit data, which makes them a very interesting choice in practice.

Replaying a series of changes from an operation log is quite doable (blog post incoming). But having a way to compress / annotate the operation stream will lead to far better performance in lots of ways. Especially as Kevin says - with CRDTs like Yjs and automerge which consider document order (not time order) as the canonical representation.

[0]: https://news.ycombinator.com/item?id=29917818

However, all the current collaborative editing apps which use Matrix operate by serialising opaque CRDT updates as Matrix events - a bit like Wave used to send blobs of base64 as OT updates over XMPP. Matrix-CRDT is cool in terms of also transporting updates over a WebRTC ephemeral transport to get lower latency (although it's missing a trick that the WebRTC looks to be signalled over websockets rather than just using Matrix's VoIP signalling to give you E2EE decentralised WebRTC signalling for free ;)

Now, Matrix already is a constrained CRDT (monotonic semi-lattice, i think?) which provides primitives for key-value and conversation-timeline storage. Our merge resolution algorithm is detailed at https://matrix.org/blog/2020/06/16/matrix-decomposition-an-i.... However, we always intended to be able to store object graphs in Matrix too - and have been experimenting with APIs for traversing a DAG of objects overlaid within Matrix's room DAG (e.g. https://github.com/matrix-org/matrix-doc/blob/kegan/msc/thre... - which isn't really about threading specifically, but could be traversing any kind of object graph).

So, what we're looking at now is (i think?) precisely what you're proposing that we do - i.e. use an existing CRDT implementation to model the collaborative evolution of an arbitrary object graph - while also snapshotting that as we go as evolutions within the Matrix room DAG. I haven't been working on it myself, and we haven't published the research yet, but I'd assume we'd do something like pass CRDT updates as Matrix EDUs (Ephemeral Data Units) between clients, with the server then maintaining or generating snapshots of views of the graph. The key thing we're aiming for is to build on the existing decentralised namespace and identity model and end-to-end encryption that Matrix already provides.

This could look something like:

* Client A negotiates a WebRTC data channel with Client B for low-latency collaboration (via standard Matrix m.call.invite WebRTC signalling, so you get E2EE and decentralisation for free)

* CRDT updates also get sent to the server via the Matrix client-server API (n.b. that in the nearish future the server may actually be running locally within the client, thanks to P2P Matrix: https://matrix.org/blog/2021/05/06/introducing-the-pinecone-...). For E2EE rooms, updates would have to be at the granularity of an encrypted event (but perhaps clients participating in the E2EE could decrypt, coalesce and re-store these).

* Server maintains a view of the object graph, letting clients navigate lazily through the tree, view it or bits of it as versioned snapshots, or start participating in the CRDT itself.

(It's worth noting that Matrix events are deliberately capped to 65KB - anything bigger than that should be persisted as a binary blob on disk. In this model it's probably fine, given you'd want events to be as small as possible - possibly even keystrokes.)

Again, this is very handwavy and I'm not actually working on it myself, but it hopefully gives a vague idea of what we're thinking about.

However, not all CRDTs are the same (actually, I think the definition is still somewhat vague - most things "eventually consistent" could be called a CRDT). Yjs is specifically designed for high-performance operations and has great support for rich-text collaboration. It also works great while offline - and you can connect different providers as you like (for example, store updates both locally in IndexedDB, and transmit messages over websocket / webrtc / and now, also Matrix). Definitely wanted to keep this flexibility.

That said, using this to stream CRDTs, and THEN federating those streams between servers would be pretty wild.

1. It would be easy to manage data sharing. Post updates to data in channels that the relevant people have access to.

2. It would be easy to debug data changes. I could see it becoming the equivalent of a terminal app but for the web.

[1] https://en.wikipedia.org/wiki/Conversational_user_interface

Web 2.0 is The Empire Strikes Back.

Web 3.0 is The Room.

It works beautifully with almost no code. I used Yjs P2P, but P2P is blocked in numerous networks.

If you want to try, with an hackernews room, but sorry it's in french : https://nosgestesclimat.fr/conférence/hackernews

So using Matrix as a backend is a great idea.

The only caveat I found to Yjs is that if you want to persist data (and not lose it when the server crashes), nothing's really plug-and-play, so I went with Supabase.

I'd probably have to re-architect the whole front- and back-end though. Although on the other hand, there will be some code editors (currently using Monaco) where this could be usable.

https://github.com/replikativ/datahike

Is an Open Source variant of Datomic.

Lambdaforge wants to eventually have this work with CRDTs.

Using the Matrix ecosystem for this is quite interesting as it solves many problems for you already.

I've thought about datomic + crdts in the past, and it seems like ":db/unique :db.unique/identity" properties would be an issue. Whether or not the transaction fails depends on the current content of the database.

I don't know if there is a way around this (while still being a CRDT) or how necessary this feature is.

https://github.com/y-crdt/y-crdt

https://github.com/y-crdt/y-crdt/tree/main/y-py

It’s called TypeCell as that’s the (yet to be announced) project both Matrix-CRDT and SyncedStore are building blocks for.

If you’re interested in tech like this and maybe also “future of programming” (those bret victor things ;) ), knowledge management software (notion, roam, obsidian) - i’m sure you’ll like it!

Discord is not just proprietary, but it also has a messy UI and bad UX.

When you auth with Matrix, is that a centralized action (one company / org behind it) or is that a decentralized action (its more a protocol for auth amongst the community of users)?

ie. Is using Matrix for auth decentralized or a different centralization?

This design is fairly similar to email where you can be @gmail, @hotmail, or @whateveryoulike.com